DOCUMENT RESUME 



ED 472 605 



CE 084 330 



AUTHOR 

TITLE 

INSTITUTION 
SPONS AGENCY 

PUB DATE 
NOTE 

CONTRACT 
AVAILABLE FROM 

PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Ananda, Sri 

Supporting High School Students through Assessment of 
Academic and Industry-Valued Skills:’ What Have We Learned? 
WestEd, San Francisco, CA. 

Office of Educational Research and Improvement (ED), 
Washington, DC.; American Youth Policy Forum, Washington, DC. 
2002-06-28 
33p . 

ED- 01 -CO-OO 12 

For full text: http://www.WestEd.org/online_pubs/ 
highschoolassessment.pdf. 

Information Analyses (070) 

EDRS Price MF01/PC02 Plus Postage. 

Academic Education; Achievement Tests; Career Education; 
Curriculum Development; Educational Certificates; Educational 
Change; ^Evaluation Methods; High Schools; *High Stakes 
Tests; * Integrated Curriculum; Measures (Individuals); 

Program Improvement; *State Standards; ^Student 
Certification; ^Student Evaluation; Test Use; Vocational 
Education 

^Career and Technical Education; Contextual Learning 



ABSTRACT 

Some fundamental assessment-related issues must be considered 
as states and localities refine and build upon existing assessment systems. 
First is the likely impact of increased high-stakes testing on industry- 
valued high school programs and their students. Data are inconclusive but 
suggest consequences of high-stakes testing on these programs and students 
warrant concerted attention for positive and negative outcomes. The second 
issue is success-to-date in using assessments beyond state-mandated, 
academic-focused tests for career-technical education and other high school 
reform efforts. The field has demonstrated a range of assessment methods -that 
can be used for particular assessment purposes, including reinforcing good 
teaching and learning and emphasizing student recognition and certification. 
The current focus on high-stakes, large-scale academic testing may deter 
attention from other purposes and forms of assessment. Third is how career- 
related and other SCANS-like skills can be incorporated into states' academic 
standards and assessments. A range of possible curriculum-embedded assessment 
tasks could help teachers effectively link academic and workforce preparation 
skills in their instruction. Core academic curriculum support documents 
featuring models of such tasks benefit all students. The fourth issue is how 
assessment helps support and improve contextual teaching and learning. 
Concrete examples are strengthening local assessments in a "mixed" state- 
local assessment model and incorporating assessment data-driven decision- 
making into program improvement efforts. (27 references) (YLB) 
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Abstract 



High school reforms that incorporate industry-valued skills — integrating academics 
with “real-world” teaching and learning — have led to important lessons for 
improving student assessment systems. Drawing on that knowledge, this paper 
addresses four assessment-related questions that need to be considered as states 
and localities refine and build upon existing assessment systems: 

• What is the likely impact of increased high-stakes testing on industry-valued 
high school programs and the students they serve? 

• What success has there been in using additional assessments — beyond state- 
mandated, academic-focused tests — for career-technical education (CTE) and 
other high school reform efforts? 

• How can career-related and other SCANS-like skills be incorporated into states' 
academic standards and assessments? 

• How can assessment help support and improve contextual teaching and 
learning? 

After addressing each of these questions based on a review of data and current state 
practices, this paper concludes with specific assessment-related recommendations to 
support industry-valued high school reform. 
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Current assessment pressures pose significant challenges to advocates of industry- 
valued reforms for high schools, reforms that integrate academics with “real-world” 
teaching and learning. Efforts to assess industry-valued standards through large- 
scale, comprehensive systems that offer portable skill certificates for high school 
students have yet to reach fruition. Meanwhile, high-stakes, academic assessment 
and school accountability increasingly dominate the policy landscape. Kane, Staigar, 
and Geppert (2001) note that in spring 2000, 40 states used statewide student test 
scores to evaluate school performance, with 20 of them basing rewards and sanctions 
on a school’s test performance. The recently enacted No Child Left Behind Act of 
2001, which requires states to meet new and far-reaching federal mandates for 
student assessment and school accountability, formalizes as a national mandate 
what the majority of states have already begun on their own. Yet, such high-stakes, 
academic assessments for high school students do not systematically and formally 
incorporate work preparation (WestEd, 2001a) and skills such as those identified by 
the Secretary’s Commission on Achieving Necessary Skills, or SCANS (1991). 

There is much support among policymakers and educators across the nation for the 
philosophy embodied in the No Child Left Behind Act of 2001. Nonetheless, there is 
also widespread concern among states and districts about how to meet the law’s 
myriad assessment and accountability requirements. The new law calls for states by 
2005—06 to implement annual testing in mathematics and reading for grades 3 
through 8 as well as testing at least once in these subjects during high school. By 
2007—08, states also need to implement testing in science at the elementary, middle, 
and high school levels. Based primarily on student performance on these 
assessments, starting in 2002-03, a state must also set up a single accountability 
system whereby all students score at the “proficient” level by the end of 12 years and 
demonstrate annual, incremental progress toward this goal. A common fear within 
high school reform circles is that the increased attention to academic, high-stakes 
testing and school accountability mandated by this new law may negatively impact 
some of the students served by high school reform programs — particularly minority 
students, low socioeconomic-status students, English language learners, and special 
education students — and constrain the programs themselves. 

Despite these new and formidable assessment challenges, high school education 
reform efforts that incorporate work-related requirements and industry-valued skills 
have led to important lessons that could serve to improve student assessment 
systems. It is important for policymakers, educators, and trainers involved with high 
school and youth programs to be aware of these lessons as well as other assessment- 
related issues that impact the youth they serve. The high school level, in particular, 
faces assessment needs that have yet to be met. Specifically, some programs are 
intensifying their work-related curriculum, thus requiring assessments that are 
reflective of this shift. Furthermore, increasing numbers of school districts and 
states are beginning to explore and undertake substantial high school reforms that 
will require additional enhancements to their assessment strategies. 
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To that end, this paper addresses some of the fundamental assessment-related 
questions that need to be considered as states and localities refine and build upon 
existing assessment systems: 

• What is the likely impact of increased high-stakes testing on industry -valued 
high school programs and the students they serve? 

• What success has there been in using additional assessments — beyond state- 
mandated, academic-focused tests — for career-technical education (CTE) and 
other high school reform efforts? 

• How can career-related and other SCANS-like skills be incorporated into states’ 
academic standards and assessments? 

• How can assessment help support and improve contextual teaching and 
learning? 

This paper poses answers to each of these questions based on a review of data and 
current state practices. The paper concludes with specific assessment-related 
recommendations to support industry-valued high school reform. 

What is the likely impact of increased high-stakes testing on industry-valued 
high school programs and the students they serve? 

There are good reasons for concern that the increase in high-stakes testing may 
have a negative impact on high school reform efforts and the students they serve. 
First, high school reform efforts serve a range of students with different learning 
and response styles. These students’ skills might not be best demonstrated through 
paper-and-pencil standardized testing. High-stakes tests are typically limited to 
multiple -choice items only or multiple -choice items plus a limited number of 
constructed-response items that require written responses (typically, short-answer 
items or essays). Other test formats (e.g., hands-on, performance-based assessments; 
portfolios) are too resource intensive to administer and score on a statewide basis 
(Ananda & Rabinowitz, 2000; Rabinowitz & Ananda, 2001). Multiple -choice testing, 
in particular, is inconsistent with the constructivist teaching and learning focus of 
many high school reform efforts. Thus, students who are benefiting from 
constructivist, contextualized learning situations may not be expected to perform as 
well on multiple-choice testing because of its emphasis on isolated, fragmented 
concepts. Moreover, many statewide academic assessment systems rely extensively 
on norm-referenced tests (NRTs) that are designed to measure how well students 
have achieved relative to each other and relative to national norms. NRTs are often 
not well-aligned to a particular curriculum and hence may not measure what a 
student has learned in the classroom. 
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Yet another concern about the possible effects of high-stakes academic testing on 
students is that such testing may deter them from fully participating in applied 
learning courses and industry-valued programs. Prior to the most recent wave of 
increased high-stakes academic testing, dechnes had already occurred in the number 
of high school students taking a concentrated number of career-technical education 
(CTE) courses. A recent study reported that the portion of students who took three 
or more courses in a single occupational program area decreased from about 33 to 25 
percent among high school graduates from 1982 to 1994 (Levesque et al., 2000). This 
trend was found to be accompanied by increases in academic course-taking and is 
also thought to be linked to increases in high-stakes academic testing (National 
Dissemination Center for Career and Technical Education, 2001). 

High school exit exams are suspected to be the measures that would have the most 
negative impact on student participation in industry-valued programs. Specifically, 
it is thought that students who do not initially pass the high school exit exam will be 
targeted for remediation in academic subjects areas and, consequently, will be 
discouraged from participating in CTE and other industry-valued programs. More 
significantly, a common fear is that the implementation of challenging high school 
exit exams will result in an increased dropout rate, especially among specific sub- 
groups of students (i.e., minority students, low socioeconomic-status students, 
English language learners, special education students). Although some research 
studies suggest a link between increased dropout rates and high school exit exams, 
such studies often do not adequately control for potentially confounding variables 
(for example, student background factors and programmatic changes, such as more 
rigorous content standards and curricula). Thus, the evidence that links increased 
dropout rates of subgroups of high school students to high school exit exams is not 
conclusive (Rabinowitz, Zimmerman, & Sherman, 2001). 

On the other hand, there is also a lack of definitive data to support the often-cited 
claim that participation in industry-valued high school reform initiatives leads to 
increases in student academic achievement. Indeed, while the available data from 
high school innovations that provide a thematic focus (e.g., career academies, career 
magnates) demonstrate higher student attendance, fewer discipline problems, and 
higher likelihood to attend college as compared with traditional high schools, there 
is little evidence of the significant increases in student achievement anticipated by 
proponents of industry -valued high school reform efforts (Katz, Jackson, Reeves, & 
Benson, 1995; Lynch, 2000). In fact, some data seem to suggest lower performance of 
students in industry-valued programs on academic achievement tests. However, in 
their study of Arizona high school students’ Stanford-9 scores, Elliot and Knight 
(2002) found that when they statistically controlled for extraneous variables (e.g., 
disproportionately large numbers of students from special population groups in CTE 
programs), apparent test score deficits for CTE students were negligible. 
Nevertheless, the authors warned that school leaders need to take care to control for 
such variables when conducting analyses of CTE programs and general education 
tracks; otherwise, CTE programs may face discrimination as the nation rushes 
ahead with implementation of high-stakes academic testing. 
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High Schools That Work (HSTW) embodies a possible exception to the lack of 
evidence of increased student academic achievement associated with industry- 
valued high school reform efforts. HSTW is designed to increase the academic 
achievement of career-focused high school students by combining the content of a 
college preparatory curriculum with CTE. Administered by the Southern Regional 
Education Board, HSTW was 1 of 24 nationwide school reform initiatives reviewed 
in a recent study by the American Institutes for Research (AIR). In its review, AIR 
found few studies of student outcomes for the 24 school reform initiatives that were 
based on rigorous research designs (Herman et al., 1999). However, cumulative 
evidence over more than a decade suggests that HSTW schools have increased 
student achievement. More specifically, a study conducted by MPR Associates under 
the auspices of the National Center for Research in Vocational Education 
demonstrated increases in HSTW assessment scores from 1996 to 1998 in reading, 
math, and science (Kaufman, Bardby, & Teitelbaum, 2000). 

In summary, the data to date on the potential impact of high-stakes testing on 
industry-valued programs and the students they serve are inconclusive. However, 
the data suggest that the consequences of high-stakes testing on these programs and 
the students being served warrant concerted attention for both positive and negative 
outcomes. 

What success has there been in using additional assessments — beyond 
state-mandated, academic-focused tests — for career-technical education and 
other high school reform efforts? 

During the past decade, states and localities have investigated and developed a 
range of assessment tools, in part spurred by national efforts and supported by the 
federal government. The School-to-Work Opportunities Act of 1994 (STWOA) calls for 
standards-based assessment that leads to portable skill certificates for high school 
students who master important industry-valued and related academic skills. 
Similarly, the National Skills Standards Act of 1994 envisions a comprehensive 
national system of standards-based assessments and certification, encompassing 
both entry-level workers as well as the full range of adult workers in major industry 
sectors. More recently, the National Association of the State Directors of Career 
Technical Education Consortium has been exploring systemwide options for high 
school end-of-course assessments to underscore career-technical education (CTE) as 
an integral part of the total education system (Wills, 2002). 

To date, the above-mentioned goals of large-scale systems of career-related 
assessments and certification for high school students still remain mostly elusive at 
the state and national levels. There are many formidable obstacles precluding states 
from endorsing and using instruments that lead to portable skill certificates. Among 
them is the difficulty of identifying any single assessment instrument that is 
appropriate within and across career majors, schools, and districts. The wide range 
of local programs with different instructional foci suggests a need for different 
assessment approaches and instruments to meet local needs. On the other hand, 
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allowing for different assessments would make it hard to achieve portability, 
whereas implementing a single, common statewide assessment would virtually 
assure the portability of skills certificates or credentials. 

Because of the difficulty in finding a single, hence fully portable, assessment 
instrument that would fit well with different programs, many states have either 
explicitly or implicitly encouraged local programs to devise their own assessment 
and certification models to support locally endorsed industry-valued models. Given 
this local flexibility, a number of promising assessment models have emerged. While 
some of these involve selected-response (i.e., multiple -choice) assessment, many 
feature performance-based assessment. Performance-based assessments require 
examinees to construct or produce a response to an assessment item or task, as 
opposed to multiple-choice testing in which examinees must select the correct 
response. At its best, performance-based assessment includes cognitively 
demanding, hands-on activities that aim to stimulate learners to think, react to new 
situations, review, revise, and evaluate their work, and communicate in verbal and 
visual ways. Examples of performance-based assessment methods include problem- 
solving scenarios, performance events, and computer simulation tasks. 

Described below are selected examples of assessment and certification models being 
developed or implemented in CTE and other industry-focused contexts. They have 
demonstrated promise in the field as alternatives for the assessment and 
certification models espoused by the STWOA and other related legislation. They 
range from specific instruments (WorkKeys), to performance-based assessment 
methods (problem-solving scenarios, projects, portfolios), to innovative assessment 
frameworks and delivery modes (national industry-endorsed credentials, computer- 
based assessment). 

WorkKeys 

Developed by ACT, Inc., WorkKeys is a system for instruction and testing of general 
workplace competencies and employability skills. It is an assessment tool that shows 
students their levels (assessment profiles) in eight foundational skills: applied 
mathematics, applied technology, listening, locating information, observation, 
reading for information, teamwork, and writing. It also provides a job analysis 
system that identifies skill levels that employees need to perform competently in 
specific jobs. While it is primarily a paper-and-pencil assessment, WorkKeys is now 
including computer-based delivery for some of its components. Nearly 14,000 
companies nationwide have used WorkKeys for initial hiring, training, and 
employment (Lynch, 2000). In the past few years, thousands of high school students 
have been tested with WorkKeys. For example, Oregon's Certificate of Advanced 
Mastery assessment includes specific local options to measure student performance 
relative to the Career-Related Learning Standards (CRLSs). WorkKeys was included 
as part of a pilot study to determine valid measurement of CRLSs. Similarly, 
Michigan's Department of Career Development has proposed using WorkKeys 
throughout the state as a measure of general workplace skills for secondary-level 
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students and welfare-to-work program participants. Illinois also has incorporated 
part of the WorkKeys battery in its state-mandated Prairie State Exam for grade 11 
(Wills & Kaufmann, in progress). 

There are many benefits to using the WorkKeys system for assessment of high 
school students’ proficiency on workplace readiness and employability skills. Most 
significantly, WorkKeys is a standardized assessment package that is based on 
extensive research. Thus, it is appropriate for use in a range of educational settings 
and has empirical, psychometric data to justify its use. However, some WorkKeys 
users have also suggested a few limitations. Specifically, WorkKeys assesses a range 
of workplace skills, from rote to more complex. As such, some teachers find 
WorkKeys not fully aligned with their curriculum which integrates workplace 
readiness and challenging academics. Others suggest that the video and audio 
stimuli for some of the WorkKeys assessment items seem removed from high school 
students experiences and interests. Despite these apparent limitations, few, if any, 
other assessment packages for workplace readiness skills are as comprehensive and 
well-researched as WorkKeys. 

Problem-solving scenarios 

Scenarios represent a popular approach to assessment of career-related skills. They 
depict complex and realistic problems in a work-related or other “real-world” 
situation. An examinee’s response to the scenario demonstrates his or her ability to 
apply previous knowledge to generate solutions to realistic problems. Table 1 shows 
an example of a scenario that requires a written response from high school students 
studying animal science. 
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Table 1: Example of a written scenario prompt in animal science 



Species Selection 

SCENARIO 

A neighbor has inherited 500 acres of 
sparsely forested range land and has 
decided to start a large animal livestock 
operation. Your neighbor doesn’t have 
much experience with livestock, so he has 
asked you to help him choose what animals 
to run on his property. 

The property includes rolling hills. The 
primary vegetation is annual grasses and 
small shrubs. There is a wooded portion of 
the property, which contains mostly coastal 
live oak. There is a seasonal, natural water 
supply. The winters are mild and wet, but 
the summers can be very hot. There is 
perimeter fencing around the property. 

[adapted from the Career-Technical 
Assessment Program (Ananda, 1999)] 



INSTRUCTIONS 

Think about animal production. Select 
which species or combination of species 
could be used, given the conditions of your 
neighbor’s property. Give reasons for what 
you suggest. 

To receive a Proficient rating on this 
task, you must show all of the 
following: 

1. Knowledge of: 

• Earning potential in animal 
production 

• Range management 

• Animal facilities 

2. Ability to propose a solution to this 
scenario 

3. Ability to communicate effectively in 
writing 



A number of skill standards projects have used scenarios to either frame the skill 
standards themselves or as a preferred method to assess mastery relative to skill 
standards. For example, the Bioscience National Skill Standards, developed by 
Education Development Center (EDC), include scenarios as routine procedures and 
unanticipated problems that the student must master (Malyn-Smith & Leff, 1997). 
These scenarios are used for learning as well as evaluative purposes. Similarly, the 
Family and Consumer Sciences National Standards, developed by the Vocational- 
Technical Education Consortium of the States (V-TECS), include scenarios for at 
least two occupational areas. The Manufacturing Linkage Project, another project 
led by V-TECS (and co-directed by the state of Indiana), has also developed 
assessment scenarios with significant industry participation (Border, 1998). 

Furthermore, Indiana is using locally developed assessment scenarios on a 
statewide basis to award individuals with Certificates of Technical Achievement. 
Certificates are awarded to students, incumbent workers, and adults whose 
performance on scenarios meet specific criteria. The Indiana Department of 
Workforce Development monitors individual assessment sites to protect the value 
and credibility of each certificate awarded (Wills & Kaufmann, in progress). 

There are many benefits to scenario assessment. Compared to other assessment 
methods that require students to construct rather than select a response, scenarios 
are fairly easy to administer and score. Scenarios are discrete and versatile tasks. 
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They can be incorporated into large-scale paper-and-pencil (as well as computer- 
based) assessments with both multiple-choice and open-ended response options. 
Using a scenario to elicit both multiple-choice and open-ended responses can result 
in more comprehensive content coverage and more efficient and reliable 
measurement. In addition to their use in large-scale testing, scenarios can also be 
used for classroom instructional and assessment purposes. 

There are some limitations to scenarios, however. First, it is difficult to develop 
useful, relevant scenarios in generic workplace readiness contexts that are 
meaningful to all students. It is much easier to develop scenarios that reflect a more 
specialized occupational context, such as marketing or home health care. Moreover, 
although scenarios are more amenable to inclusion in large-scale assessment than 
many other performance-based assessment options, scenarios are still much more 
expensive to develop and score than typical multiple-choice items. 

Project assessments 

A project is an in-depth, hands-on exploration of a topic, theme, idea, or activity, 
resulting in a product, performance, or event for assessment (Katz & Chard, 1989). 
The objective of a project is for the student to delve more deeply into and learn about 
a topic of interest. A project takes place over a substantial period of time (e.g., 
weeks, months) and represents the best of what a student can do, given constructive 
feedback and opportunities to revise his or her work. As such, projects focus on 
depth of knowledge and result in substantial work products. 

Although projects are considered an innovative assessment method, many teachers 
have used a form of project assessment at one time or another — particularly at the 
high school level. Projects are a powerful tool for assessing a blend of hands-on skills 
and challenging academic content because they require students to demonstrate in- 
depth content knowledge, evaluate their own work, solve problems, plan and carry 
out complex activities, and (often) communicate findings to an audience. The senior 
project is a form of assessment that is commonly used in high schools across the 
country. Typically adopted as a requirement for high school graduation, almost all 
senior projects include background research, a research paper, development of a 
product related to the paper, and an oral presentation of the research and product to 
an audience. Table 2 provides brief descriptions of senior projects. 
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Table 2: Examples of senior project ideas 



Neighborhood Project: Student identifies a topic of concern to his or her 
neighborhood, conducts a neighborhood survey on attitudes toward that topic (e.g., 
a new freeway construction measure on an upcoming city election), analyzes the 
results, and summarizes their implications. 

The Plant Science Experiment: Student conducts research on the factors 
contributing to the growth of a rare plant, develops some hypotheses that can be 
tested, designs a plant experiment to test these hypotheses, then summarizes 
findings on a poster board and in a written report. 

Computer Software Manual : Student investigates computer software questions 
that are most frequently asked by beginning users and develops a software 
reference manual to help users find answers to their questions. 

The Infomercial: As a group project, students research the infomercial trend, select 
a product to advertise, and create a videotape infomercial. 



The use of project assessments as an integral part of teaching and learning activities 
can have many effects on student learning and teacher practices (Long & Crepeau, 
2000). On the positive side, students can increase and deepen their knowledge and 
skills on a particular topic of interest to them. Many students show increased 
autonomy and resourcefulness as they engage in active, rather than passive, 
learning. High schools with senior projects also report higher rates of homework 
completion, increased self-reflection, and increased student competence in writing 
and research skills. At the same time, project assessment can be difficult and 
frustrating for some students who are not accustomed to an assessment that 
requires a high level of initiative and commitment. Moreover, in practice many 
senior projects tend to sacrifice academic rigor for showy presentations. This is 
especially true during the early years of project assessment implementation when 
students are not yet accustomed to projects as a vehicle for showcasing their 
knowledge and skills on key academic and industry-valued standards. 

There are benefits and drawbacks associated with project assessment not only for 
students, but also for teachers. Some teachers welcome the shift in their roles from 
lecturer to mentor, coach, and advisor that accompanies incorporation of project 
assessments into the curriculum. For others, this shift is uncomfortable and 
somewhat difficult. Moreover, certain logistical aspects of project assessment can be 
overwhelming to teachers, such as identifying and bringing in a larger audience 
(parents, other teachers, community members) to watch students present their 
projects. Nonetheless, project assessments have great potential for integrating 
academic and hands-on learning — they help create and foster an interdependence 
between learning that is hands-on and minds-on (Long & Crepeau, 2000). 
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Portfolios 



A portfolio assessment involves the structured collection of student work that 
documents students’ application of knowledge and skills in a variety of authentic 
contexts. In contrast to projects that typically require students to produce one 
product related to a few standards or themes, portfolios generally require a variety 
of student work related to multiple standards or themes. Therefore, portfolio 
assessment can usually provide a more comprehensive view than projects of 
students’ standards-based knowledge and skills (Ananda, 2000). 

There are many examples of portfolios in place today across all levels of the 
education system. Writing portfolios abound at the elementary, middle, and high 
school levels. Math portfolios, although less popular than writing portfolios, are used 
not only as an assessment tool, but also to encourage students to work with and 
engage in mathematics in meaningful and nontraditional ways. 

Although the use of portfolios to promote academic content learning has received the 
lion’s share of attention in education circles, many career-related portfolio models 
have also been developed over the last 12 years or so. Michigan’s Employability 
Skills Assessment Kit (ESAK) is considered a pioneer effort in career-related 
portfolios for secondary-level students (Michigan State Board of Education, 1993). 
Although no longer in statewide use, it is briefly described here because it was one of 
the first career-related portfolio models to gain widespread visibility and recognition. 
This portfolio model was designed to assess students’ attainment of 12 employability 
skills benchmarks and provide feedback to students about their work readiness. 
ESAK used a “legal approach” — students were presumed “innocent until proven 
skilled.” They acted as their own “prosecuting attorneys” and gathered evidence to 
prove they were skilled, basing their “case” on evidence related to the set of 12 
employability skills benchmarks. 

A career-related portfolio is the centerpiece of California’s Career-Technical 
Assessment Program (C-TAP). C-TAP’s original purpose was to certify and formally 
recognize students demonstrating mastery of important career-technical and related 
academic skills consistent with California’s Model Curriculum Standards for 
programs in Agriculture, Business, Health Careers, Home Economics, and 
Industrial and Technology Education. Because it was intended as part of a statewide 
skills certification system, the C-TAP portfolio is somewhat prescriptive in 
comparison to other high school portfolio models. It consists of five required sections 
(table of contents; letter introducing the portfolio; career development package of 
resume, completed college or job application, and letter of recommendation; work 
samples; and writing sample) and one optional section (evaluation of employability 
skills by supervisor). Beginning in the mid-1990s, thousands of career-technical 
students in California produced C-TAP portfolios. However, because of the high cost 
of large-scale implementation, coupled with California’s move away from statewide 
performance -based assessment, C-TAP never was implemented statewide as a skill 
certification system (Ananda, 1999). 
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Despite the fact that the C-TAP portfolio was never incorporated into a statewide 
system, the C-TAP model and variants of it took hold in districts and schools 
throughout California as well as in Arizona and Oregon. Specifically, the C-TAP 
model gave rise to the Career Preparation Assessment (CPA), a portfolio designed to 
measure learning relative to generic workplace readiness skills (Personal Skills, 
Interpersonal Skills, Thinking and Problem Solving, Communication, Employment 
Literacy, and Technology Literacy) and academic standards. This more generic 
measure of workplace readiness was aimed at a larger cross-section of high school 
students, including those involved in career-technical education, career academies, 
and school-to-work, as well as students not involved in such programs. As more and 
more sites began using the CPA, they sought to tailor the general model to their own 
local needs. Hence, CPA evolved into the Custom Portfolio (CP), which allows for 
some variation from the general model, while maintaining the advantages of a 
largely standardized portfolio model (e.g., use of common portfolio scoring rubrics, 
use of common exemplars of student work). 

The evolution of the C-TAP portfolio illustrates some of the strengths and 
limitations of the portfolio as an assessment tool. A major strength is its inherent 
appeal to a range of programs that seek to integrate academic and “real-world” 
learning. Numerous sites across California, Oregon, and Arizona were attracted to 
C-TAP and its variants. In many ways, the career-related portfolio represents an 
ideal curriculum-embedded assessment because of its ability to address multiple 
standards in interesting, meaningful, and compelling ways. Like projects, students 
typically take pride in their portfolios, demonstrating increased autonomy, self- 
reflection, and initiative as they develop their own unique collection of work. 

However, as the C-TAP experience demonstrates, portfolio models in general have 
not proven sustainable for statewide assessment purposes. They are very expensive 
to administer and score, and raise a host of logistical issues (e.g., where to store 
portfolios). Use of portfolios also requires significant professional development for 
teachers. And sometimes portfolios do not survive public scrutiny. That is, some 
factions in the general public consider portfolios as intrusive, non-rigorous 
measures. Nevertheless, for local instructional and assessment purposes, portfolio 
assessments fill a niche not easily addressed by other methods. 

Incorporation of national industry-endorsed credentials 

Given the significant development and administration costs associated with new 
assessments that attempt to address industry-valued skills, some states have 
decided to meet assessment needs by drawing directly from industry. For example, 
Virginia has developed a multifaceted process to incorporate industry-validated 
assessments into the state assessment system and provide students with portable 
credentials. Substantial effort has been made in systematically identifying the 
academic requirements embedded in nationally validated skill standards 
certification programs. Using an instrument called the Snyder Taxonomy, made 
available through the Vocational-Technical Education Consortium of the States (V- 
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TECS), Virginia cross-referenced (or cross-walked) industry standards with its own 
state academic standards. Although this effort required some investment in 
software, it avoided substantial assessment development and implementation costs. 

In short, Virginia’s approach is to address academic standards through their state 
tests and ensure linkage to industry requirements by using the assessments 
developed by industry. This approach has provided additional benefits for career- 
technical education programs. The state, through the documentation of academic 
requirements in materials based on industry standards, has also been able to 
expand dual enrollment opportunities for some students in post-secondary 
institutions. While this assessment approach is certainly promising, one possible 
limitation relates to its total reliance on industry-endorsed credentials for 
measurement of SCANS-like skills. Just as high-stakes academic assessments may 
not represent the most instructionally sensitive measures of student learning, 
industry assessments may be too focused on a single occupation to reflect the 
broader, industrywide focus of many high school programs. 

Computer-based assessment 

Some of the performance-based assessment methods described above (projects, 
portfolios) have experienced success at the local level, but have proven harder to 
implement as high-stakes assessments at the state level because of cost and 
administration considerations. Now, with the advent of computer-based assessment, 
comes the potential for vastly improving how assessments are delivered and scored 
as well as the quality of information they generate. There are several advantages of 
computer-based assessment over the traditional paper-and-pencil mode of 
assessment delivery. A computer-based delivery system makes it possible to include 
a range of assessment methodologies that allow for dynamic interaction between 
students and assessment items, including different stimuli for questions that are 
difficult or cost-prohibitive to use with a paper-and-pencil assessment system (e.g., 
complex diagrams, color photographs, audio and video clips). With use of computer- 
adaptive models, assessment administration time is cut short and the assessment 
process is better targeted to the individual student’s needs because each student is 
presented with assessment items tailored to his or her ability level. Further 
efficiencies are achieved because computer-administered tests require no printing 
and shipping of test booklets and answer forms. Moreover, computer-based tests can 
produce instantaneous results, even for assessment programs that require student 
writing because of new breakthroughs in artificial intelligence and other models that 
allow timely computer scoring of essays. (For a full discussion of the benefits and 
drawbacks of computer-based assessment, see Rabinowitz & Brandt, 2001.) 

There are many possible variations of computer-based assessment models, many of 
which are being explored and developed to measure career-related and other 
SCANS-like skills. For example, the Sales and Services Voluntary Partnership, 
facilitated by the National Retail Federation with support from the National Skill 
Standards Board, is in the early stages of implementing a computer-based 
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assessment to measure customer service skills. This assessment uses a multiple- 
choice item format exclusively, but includes video and audio clips as stimuli for some 
of the questions. Another national skill standards-related project is developing an 
online, computer-adaptive assessment based on the National Health Care Skill 
Standards. With a U.S. Department of Education Grant, the National Consortium 
on Health Science and Technology Education has partnered with Brainbench to 
develop this assessment, which features the adaptive capability of branching to 
harder or easier assessment questions based on the individual examinee's answers 
to previous questions. 

In conclusion, the field has demonstrated a range of assessment methods that can be 
used for particular assessment purposes. Some of these purposes are about 
reinforcing good teaching and learning, whereas others emphasize student 
recognition and certification. Unfortunately, the current focus on high-stakes, large- 
scale academic testing may deter attention from these other purposes and forms of 
assessment (American Youth Policy Forum, 2000). 

How can career-related and other SCANS-like skills be incorporated into 
states’ academic standards and assessments? 

It is commonly acknowledged that the so-called “soft” skills, including work 
readiness and career-related skills (e.g., those identified by the Secretary's 
Commission on Achieving Necessary Skills, or SCANS) are neither broadly nor 
systematically covered in the vast majority of state core academic standards and 
assessments (WestEd, 2001a). Instead of being addressed in states' core student 
assessment systems, workplace readiness and career-related skills have typically 
been relegated to third-party assessments, such as state licensure exams offered by 
the National Occupational Competency Testing Institute; industry-sponsored 
certification exams, such as Automotive Service Excellence; and more recently, 
workplace readiness assessments, such as WorkKeys (Wills, 2002). 

However, some state systems are exceptions. They include workplace readiness and 
career-related skills in their state core academic standards and assessments. A few 
of these states are listed below. 

California. Although workplace and other soft skills are not addressed in 
California's core assessments for grades 3 through 8, the statewide assessment 
system includes end-of-course/end-of-program exams at the secondary level in 
academic areas as well as five broad CTE areas (Agriculture Core, Computer Science 
and Information Technology, Food Service and Hospitality, Health Care Core, 
Technology Core). These CTE exams are part of the Assessments in Career 
Education program. 

Kentucky. This state assesses Practical Living/Vocational Skills core content for all 
students as part of its school assessment and accountability system. 
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Maryland. Maryland’s state assessment system includes World of Work and 
Survival Skills. However, plans are currently underway to replace the Maryland 
high school exit exam (where these career- related skills are covered) with end-of- 
course exams in English, government, algebra, geometry, and biology (Education 
Commission of the States, 2000). 

Virginia . As previously described, Virginia incorporates national certification 
programs and their assessments into its overarching state assessment framework, 
allowing students to earn portable credentials. 

Why aren’t more states incorporating workplace readiness skills into their core 
standards and assessment systems? One reason is that while there has been 
growing public concern for the need to increase student learning with respect to 
challenging academic standards, there has been no comparable outcry from the 
general public for raising student achievement relative to workplace readiness or 
other SCANS-like skills (WestEd, 2001a). Nevertheless, employers in many states, 
along with like-minded policymakers and educators, continue to voice concern about 
the lack of workplace readiness competencies or SCANS-like skills. Many are 
advocating for formally incorporating SCANS or SCANS-like skills into core 
academic assessments so that all students are assessed on these important skills. 

However, “adding on” SCANS-like skills may place a significant burden on these 
statewide assessments whose primary purpose is to measure student learning 
relative to academic standards. Moreover, it is not clear that the resources and 
widespread support needed to effect such a substantial change in these assessments 
would be forthcoming from the federal government, the education community, or the 
general public. Even if support and resources were available, it would take years to 
overhaul the existing state academic assessments in order for them to adequately 
address workplace readiness skills. First, the standards and curriculum upon which 
the assessments are based would need to formally incorporate SCANS-like skills. 
Then, students would need sufficient exposure to these new standards and 
curriculum before being subjected to high-stakes statewide testing on this new 
content. A final consideration is that the multiple-choice and short-answer format of 
statewide academic assessments limits their ability to adequately cover key aspects 
of SCANS-like skills, such as teamwork, exercising leadership, and other 
interpersonal skills. For all these reasons, it may be unrealistic to expect that state 
academic core assessments serve as the primary vehicles for assessing SCANS-like 
or other career-related skills (WestEd, 2001a). 

However, these tests can and should serve to reinforce SCANS skills in a limited, yet 
purposive fashion. In fact, a recent analysis of a statewide academic assessment 
system showed that some academic assessment items are already cast in a 
workplace (or other “real-world”) context or address SCANS skills (WestEd, 2001a). 
Although the study found that coverage of SCANS skills on the state tests was 
limited and uneven, it noted that academic assessment instruments could be infused 
with such items in a systematic way by building them into the assessment 
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blueprints (i.e., assessment plans) for academic tests. Described below are three 
examples of items on statewide academic assessments that measure student 
learning relative to academic standards and highlight workplace contexts or 
reinforce SCANS skills. 

Example 1: Multiple-choice reading comprehension item 

Table 3 shows an example of a functional reading passage — that is, one that contains 
procedural information for real-world tasks. Specifically, the passage details 
procedures for answering a business telephone call, followed by items testing students’ 
comprehension of those procedures. Note that the first two items require students to 
comprehend details from the passage. In contrast, the third item requires students to 
draw an inference from what they read about the best reason for having telephone 
answering guidelines. As a group, this cluster of items is both cast in a workplace 
context and shows a partial match to the SCANS Reasoning skill. 

Example 2: Multiple-choice mathematics item 

Table 4 shows an item adapted from a high school exit exam. It is a carpentry 
problem that asks students to determine the appropriate factor for sketching the 
measurements of an entertainment center to scale. Students must use the 
accompanying graph to figure out a solution to the problem. Like the first sample 
item, this math item is both cast in a workplace context and requires the student to 
use the SCANS Reasoning skill. 
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Table 3: Reading comprehension prompt and multiple-choice items 



Procedures for Answering a Call in Your Workplace 

Do you know what to say when you answer the phone? Follow these guidelines: 

• Identify yourself. State your workplace and your name. Say something like, “Apple 
Appliances, Sam speaking.” 

• Speak clearly and slowly because the caller cannot see you. 

• Give the caller your full attention. Stop what you are doing before you answer the 
phone. Don't work and talk on the phone at the same time. 

• Listen carefully to what the caller says. 

• Take notes on your message pad. Write the caller s name and why he or she is 
calling. 

• Ask questions if you aren't sure what the caller wants. 

• At the end of your call, summarize what the caller has said. You can use your 
notes. 

From Communication , Steck-Vaughn Company 

1. According to the passage, what is the first thing you should do when you answer the 
phone or make a call? 

A. take careful notes on your message pad 

B. invite the caller to identify himself or herself 

C. give the caller your full attention 

D. say who you are and where you work 

2. The passage says that because your caller cannot see you, you need to 

A. identify yourself. 

B. ask polite questions. 

C. speak slowly and clearly. 

D. make a good impression. 

3. What is the best reason to have guidelines for answering a business call? 

A. to monitor the phone etiquette of employees 

B. to increase business through customer satisfaction 

C. to summarize a caller's request for assistance 

D. to practice taking complete notes over the phone 

Note: the correct answers are: 1-D ; 2-C; 3-B. This example was adapted from released 
items from the Nevada Proficiency Examination. 
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Table 4: Mathematics multiple-choice item 



A carpenter sketched to scale the pattern for the decorative top for a new 
entertainment center . If the entertainment center is to be 72 inches wide , how 
many decorative tops could be cut from a 4' x 8’ sheet of plywood? 




A. 1 

B. 2 

C. 4 

D. 12 

Note: the correct answer is B. This example was based on a released item from 
the Kentucky Core Content Test. 



Example 3: Performance task in laboratory science 

While the first two examples are multiple-choice items, this final example is a 
performance task in laboratory science. It is an excerpt from a released item from one 
of California's end-of-course high school assessments, the Golden State Exam (GSE) in 
Laboratory Science. Besides multiple -choice items, GSE science exams feature 
multiple-part performance tasks. This example is cast as a problem farmers face as 
they endeavor to produce healthy rice pdants. It requires students to apply their 
understanding of biology and genetics to explain disease resistance in rice. Moreover, 
students must use the scientific method to carry out and record observations and 
results of this lab task and, based on their findings, provide recommendations to 
farmers. Table 5 shows the first three questions of this nine-part task. 
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Table 5: Laboratory science performance task 



Genetic Relationships in Rice Plants 

Some farmers have crossed plants that are resistant to the bacteria with those that are not. After crossing the 
plants, they counted the number of offspring that had resistance to the bacteria (healthy) and those that did not 
(diseased/dying). Their results are shown below. 



Cross 

Parent plants (P) Resistant to disease X Not resistant to disease 



First generation (Fi): 



Number of plants resistant to 
disease (healthy) 


Number of plants not resistant to 
disease (diseased/dying) 


0 


987 



1 . The farmers were disappointed with the results of the cross. Use your understanding of genetics to provide 
a possible explanation for these results. 

A child of one of the farmers had been learning about genetics in biology class and suggested that it 
was important to continue the experiment by crossing the offspring (Fi) plants with each other. The 
results are shown below. 

Cross 

Offspring (Fi) Resistant to disease X Not resistant to disease 



Second generation (F 2 ): 



Number of plants resistant to 
disease (healthy) 


Number of plants not resistant to 
disease (diseased/dying) 


227 


760 



2. Use your understanding of genetics to explain these results. 

3. Describe a process the farmers might use to obtain a crop of rice that is entirely resistant to disease. 
Explain why this process would be successful. 

Note: This is an excerpt from a released item from the Golden State Examination in 
Laboratory Sciences. 
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Compared to the previous examples of multiple-choice items, this performance task 
shows stronger connections to SCANS skills, due largely to the open-response format 
of this task as compared to the selected-response nature of multiple -choice items. 
Specifically, this excerpt of the performance task: 

(1) is cast in a workplace context ; and 

(2) shows a full match to the SCANS skill of Reasoning in Questions 1 and 3 and a 
partial match to the skill of Acquires and Evaluates Information in Question 2. 

The point of showing these sample items is simply that more of such items could be 
included in state academic core assessments in a purposeful, rather than 
happenstance, fashion. In order to accomplish this, the assessment blueprints for 
state core assessments should be updated to specify that some percentage of items 
should meet one or both of the following criteria: 

(1) portray workplace or other “ real-life” contexts; and 

(2) reinforce SCANS skills (or other similar skills that overlap with academic skills), 
such as Reasoning, Acquires and Evaluates Information, and Creative Thinking. 

State-sponsored content and development committees that include academic and 
CTE educators as well as community and business representatives should determine 
the appropriate percentage of items in a state assessment that should meet the 
above criteria for each content area and grade level. 

In addition to infusing workplace readiness and other SCANS-like skills into 
existing statewide assessments, it is equally important that evaluation criteria be 
developed to measure these skills. To effectively assess such skills within existing 
test instruments, evaluation criteria and rubrics should be amended to explicitly 
specify the targeted skills. For example, to accurately evaluate student performance 
on the SCANS dimension of Improves and Designs Systems, the work product will 
need to be assessed in terms of how well the student has demonstrated this ability. 
Reliable assessment of such skills involves explicit incorporation of skills into the 
scoring system as opposed to merely assuming that a particular assessment item 
implies the use of certain skills. 

Before incorporating SCANS-like skills into formal academic assessments, these 
skills must be infused into the core academic curriculum. There are many different 
ways of doing so. One way would be via curriculum-embedded instructional and 
assessment tasks that integrate learning of academic content and SCANS skills. 

Such curriculum-embedded tasks either could be added to existing academic 
standards and curriculum documents or be featured in new supplemental 
curriculum support materials. Tasks that explicitly integrate academic and 
workplace skills are much more commonly found in career-technical curricula than 
in core academic curricula (WestEd, 2001a). 
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Curriculum-embedded tasks that integrate academic learning and workplace 
preparation could take many forms. For example, a middle or high school language 
arts task could ask students to write a letter introducing themselves and their skills 
to prospective employers. A project assessment is a curriculum-embedded task of 
larger scope. An example would be a group community service project, such as 
organizing and executing a fund-raising drive to beautify a neighborhood park. 
Group projects of this sort require students to use their knowledge of civics as well 
as key SCANS interpersonal skills, such as teamwork and leadership. 

Arguably, portfolios represent the most extensive type of curriculum-embedded 
assessment task. The aforementioned Career Preparation Assessment (CPA) 
portfolio is an example of a portfolio model that has been tailored to local needs and 
then incorporated into the curriculum of several districts and schools in California, 
Arizona, and Oregon. The CPA (and its most recent incarnation, known as the 
Custom Portfolio) is designed to measure generic workplace readiness standards 
that are SCANS-like, including Personal Skills, Interpersonal Skills, Thinking and 
Problem-Solving Skills, Communication Skills, Occupational Safety, Employment 
Literacy, and Technology Literacy. In tailoring the CPA to their own specific 
education objectives and needs, many local sites have chosen to infuse higher levels 
of academic learning with workforce preparation by explicitly linking portfolio 
components to their own challenging academic standards. 

In summary, there is a range of possible curriculum-embedded assessment tasks 
that could help teachers effectively link academic and workforce preparation skills 
in their instruction. By featuring models of such tasks in core academic curriculum 
support documents, all students will benefit from meaningful reinforcement of 
academic learning through “real-world” applications. 

How can assessment help support and improve contextual teaching and 
learning? 

Assessment can either support contextual teaching and learning or hinder it. The 
significant mismatch between existing state assessment systems and contextual 
teaching and learning represents a major obstacle. Most state assessments consist 
mainly of decontextualized multiple-choice and short-answer items that address 
academic core content, whereas contextual teaching and learning emphasizes hands- 
on, integrated learning of academic and real-world skills. Until state assessment 
systems embrace tasks that measure SCANS-like and academic skills in context, 
large-scale core assessments will not be supportive of contextual teaching and 
learning. Nonetheless, there are specific ways in which assessment can be 
supportive of contextual teaching and learning. Described below are two strategies 
for fostering such support. 
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“Mixed” state-local assessment model 

One strategy is to design an assessment framework that combines local and state 
assessments (Rabinowitz & Ananda, 2000). As statewide assessments focus 
increasingly on high-stakes student and school accountability concerns, they tend to 
rely increasingly on more conservative assessment methods, primarily multiple- 
choice tests. The draw of these instruments is their ability to reveal patterns of 
relative strengths and weaknesses across large groups of students in a valid, 
reliable, and efficient manner. However, they generally do not yield specific-enough 
data to use in targeting instruction for individual students. Moreover, for the 
reasons cited above, it is somewhat unreasonable to assume that such measures 
could be the primary vehicle for assessing SCANS-like skills. 

Nevertheless, the limitations of state academic assessments suggest an important 
function for local assessment: assessing SCANS-like skills in contextually 
appropriate ways and providing diagnostic information about what students do well, 
where they are having difficulty, and how instruction might be adjusted to address 
their specific needs. Because they are not constrained by the same issues as state- 
level programs, local assessment programs have greater potential for generating 
such measures that are specifically tailored to local needs. Thus, the incorporation of 
local assessments into an overall statewide framework could strengthen the 
statewide system. However, it is important to note that under a mixed state-local 
assessment model, the state maintains critical responsibilities. Specifically, the state 
must provide oversight and monitoring of local efforts as well as professional 
development support, technical assistance, and dissemination of information on best 
practices. 

There is precedent for a model that combines features of local and state assessments 
to measure SCANS-like skills. Oregon is designing such an approach for its 
Certificate of Advanced Mastery assessments. Under this model, substantial 
responsibility for assessment of SCANS-like skills would be delegated to local 
programs and schools, with the state's role being that of oversight, approval, and 
technical assistance. That is, the state would provide parameters and models for 
assessments that measure SCANS-like skills, allowing schools and local programs a 
choice for which assessment tools to select or develop. Specifically, the Oregon 
Department of Education would develop a list of approved assessment methods 
ranging from checklists up through more formal, curriculum-embedded methods 
(e.g., senior projects, portfolios). The list would include sufficiency criteria against 
which potential methods and instruments would be evaluated. The department of 
education would include approved instruments within each category of assessment 
methods. These instruments, reviewed against the criteria, could be locally 
developed or commercially available. 
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Data-driven decision-making 



Despite the mismatch between state core academic assessments and contextual 
teaching and learning strategies, the data from such assessments should not be 
ignored. In fact, the results from statewide and local assessments should be used to 
inform and guide curricular and instructional decisions — including programs that 
emphasize contextual teaching and learning. Data-driven decision-making is a 
strategy that can help support and improve contextual teaching and learning. It 
refers to using quantitative and qualitative data to guide programmatic and 
instructional decisions. In a data-driven school or program, educators collect and use 
data on an ongoing basis to gauge their instructional practices and refine their 
strategies, as needed. 

Data for program and instructional improvement should be broad-based and include 
indicators about students, teachers, parents, and others. Although assessment data 
are generally the major focus of analyses, consideration of the “whole picture” is 
necessary to make informed decisions. This entails interpretation of multiple data 
elements together (WestEd, 2001b). 
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Figure 1: Using data on the road to instructional improvement 



1 . Are we ready? 



2. What data do we have and need? 



3. Who are we? 








4. What is our student performance 
and achievement? 






5. Why is our achievement as it is? 







6. What are our priorities? 






7. What are effective strategies to meet our 
priorities? 








8. What is our plan for program and 
instructional improvement? 



9. Are we doing what we said we would do? 



10. How effective was our plan? 



Figure 1 shows the major steps (or questions of interest) in a data-driven decision- 
making framework. Within each step, there are specific activities that program or 
school teams use to answer the questions posed. Through this systematic process, 
teachers working together and with others identify their programmatic data needs, 
engage with and understand the implications of their student assessment data 
results, identify necessary adjustments to instruction to meet goals and priorities, 
and gauge the effectiveness of the adjustments they make. In order to properly 
implement the decision-making process, it is important to establish both formal and 
informal feedback loops. The more fully developed data-driven systems include 
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multiple data sources that combine assessment and non-assessment indicators to 
make decisions about systems, instruction, and individual students. 

The strategies described here — strengthening local assessments in a “mixed” state- 
local assessment model and incorporating assessment data-driven decision-making 
into program improvement efforts — are two concrete examples of ways to make 
assessment an integral and meaningful component of contextual teaching and 
learning reforms. Note that both strategies rely heavily on efforts at the local, rather 
than at the state and national levels. Indeed, the various assessment-related 
challenges faced by high school reform initiatives call for concerted, but 
differentiated, actions at the national, state, and local levels. 

Concluding remarks and recommendations 

This paper has argued that industry -valued high school reform efforts face various 
assessment-related challenges as well as promising options. While states and 
localities have made progress over the past decade in expanding contextual learning 
choices for all students (e.g., expanding career academies and service learning, 
creating small learning communities, and using communities as classrooms), a major 
source of tension is how to ensure these choices are consistent with accountability- 
driven assessments. Despite the emergence of promising assessment practices 
described in this paper, the assessment tools typically being used today to measure 
student achievement in the context of school accountability may not be sufficiently 
valid to capture the learning that occurs — particularly for students with different 
learning and response styles. This situation is being exacerbated by the virtually 
exclusive emphasis in federal education reform legislation concerning high-stakes 
testing and school accountability on the “basic” skills of reading and math. 

What roles should be assumed at the local, state, and national levels to improve 
assessment systems for students in industry- valued high school reform programs? 
Based on the research conducted for this paper, specific recommendations are 
offered for roles that can be assumed and actions that can be taken at the local, 
state, and national levels. Some of these have been previously described, however, 
all warrant further delineation, particularly with respect to what can realistically be 
accomplished over the short and long terms. 

(1) Local responsibility: Incorporate SCANS-like skills into local assessments that 
are supported and monitored by the state. A major observation described in this 
paper is that assessments selected, developed, and implemented at the local level 
are typically subject to fewer constraints than those at the state level. As such, 
local assessments have greater potential for innovation beyond the conservative, 
but efficient, multiple- choice assessment format. Moreover, local assessments 
have greater potential for measuring SCANS-like skills in contextually 
appropriate ways, and in the process, providing useful diagnostic information on 
individual students. Finally, an assessment system that prominently features 
locally determined assessments can better reflect local instructional practices 
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and priorities. For example, although career-technical portfolio assessments may 
be unwieldy to administer and score at the state level, they can be used 
effectively at the local level to reinforce and measure integrated academic and 
industry-valued learning. Likewise, high school programs may choose to 
incorporate project-based assessment or computer-based assessment, depending 
on their particular program foci or student needs. 

While this recommendation argues that a strong local assessment component is 
an integral part of any comprehensive high school assessment system, it also 
recognizes the need for state oversight and support to ensure the viability and 
quality of the local component. An appropriate state role in support of local 
assessment should include technical assistance, fiscal support, and monitoring. 
Moreover, state involvement can help facilitate some degree of comparability of 
assessments across localities. Oregon's proposed assessment model for its 
Certificate of Advanced Mastery, which allows localities to select from state - 
approved assessment methods to measure industry-valued skills, represents a 
concrete example of an assessment framework that strives to balance local 
autonomy with across-state comparability. 

Because many local programs and schools already have specific assessments of 
workplace readiness skills in place, this proposed model of local assessments that 
are supported at the state level represents a realistic and attainable framework 
for high school assessment that states should consider. Costs incurred by the 
state should be moderate as compared to designing and implementing a valid, 
reliable, and legally defensible statewide assessment of workplace readiness 
skills from scratch. Moreover, implementing such an assessment system could be 
accomplished in a relatively short amount of time (e.g., within a few years). 
Preparation for full implementation could include states and localities 
researching viable assessment methods, localities pilot-testing these assessment 
methods, and the state establishing a network for support, monitoring, and 
technical assistance (including facilitating the development of local consortia). 

(2) State responsibility : Refine state core academic assessments to reinforce SCANS - 
like skills. In order to make large-scale assessment at the state level feasible, we 
must be willing to sacrifice some degree of validity in assessment of workplace 
readiness skills for technical adequacy and efficiency. Obviously, assessing “real- 
world" skills in “real-world” contexts is a more valid approach than assessing 
such skills out of context. However, if the goal is to incorporate SCANS-like skills 
in state assessment programs for all students, some adjustments will be needed 
to accommodate the large-scale, statewide assessment constraints. As previously 
described, this means relegating the more ambitious performance-based 
assessment methods, such as portfolios and projects, to the local level. 
Furthermore, it also means “settling” for more decontextualized measures at the 
state level, including multiple -choice and short-answer item formats. 
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Over the short-term, however, it may not be realistic to expect states to 
restructure their high-stakes, core academic assessment systems to serve as the 
primary means of assessing workplace readiness skills. As previously alluded to, 
the vast majority of state core assessments are not currently set up to measure 
workplace readiness skills, and there is currently insufficient political will to 
drastically overhaul state core assessments to measure these skills. Even if the 
will and resources were available, it would take years to fully realize such 
overhauls of core state assessments. 

Nonetheless, states could and should work immediately to highlight and 
reinforce SCANS-like skills on these assessments. This paper includes examples 
of items that measure core academic skills, while also reinforcing SCANS-like 
skills presented in workplace and other “real-world” contexts. Infusion of items 
that reinforce workplace readiness skills could be phased into ongoing 
assessment item development and replenishment activities. This approach would 
not be likely to face public opposition because it does not detract from the major 
function of core assessments to measure core academic skills. In fact, this 
approach has the potential to strengthen and complement the measurement of 
academic skills by encouraging the inclusion of items that are contextually 
appropriate and cognitively demanding. 

While highlighting and reinforcing SCANS-like skills on state academic core 
assessments is a realistic and worthwhile objective, it falls short of the 
overarching and longer-term goal of formally incorporating workplace readiness 
skills measurement into statewide assessment systems. In order to formally and 
comprehensively incorporate SCANS-like skills measurement, we must first 
build a compelling case for educators, policymakers, and the general public to 
recognize that SCANS-like skills matter and make a difference in student 
performance and achievement. 

(3) National responsibility: Provide leadership and support for instruction and 
assessment of SCANS-like and contextualized skills. Strong national leadership 
and support are essential to improving assessment systems for high school 
students. Just as the state should assume a substantive role in the improvement 
and support of local assessment systems, national leadership and support are 
needed to help states, in collaboration with industries, develop viable statewide 
assessment systems that effectively address workplace readiness skills. Such 
systems could build off of promising approaches developed through local and 
state efforts, such as incorporating industry-sponsored assessments in the 
overall state core assessment system or adopting a statewide system of end-of- 
course high school exams, including exams that cover workplace readiness skills. 
Whatever the particular form taken by specific state assessment systems, 
national leadership is necessary to continue building the infrastructure that 
supports connections between industry and education, including the 
collaborations necessary to ensure high quality assessment systems. 
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National leadership is also needed to give voice in support of realistic, yet 
meaningful, ways for workplace readiness skills to be included in state and 
national assessment systems, particularly at the high school level. This means 
aggressively and strategically “making the case” for the importance of 
incorporating SCANS-like skills into high school curriculum and assessment. 
Moreover, it means following the development of key federal legislation that 
affects the design of state and national assessment systems (e.g., Elementary 
and Secondary Education Act) to help ensure that such legislation supports 
incorporating workplace readiness skills into assessment systems for high school 
students. The U.S. Department of Education’s Office of Vocational and Adult 
Education, among others, would be a logical candidate for assuming a national 
leadership role in this regard. 

Finally, national leadership is essential in order to enhance our existing 
knowledge base about the relationships among contextual teaching and learning, 
assessment methodology, workplace readiness, and academic achievement. Much 
of the literature over the past decade has argued eloquently for the importance of 
students acquiring workplace readiness as well as academic skills. Nevertheless, 
we still lack “scientifically based research” (a phrase now regularly used in 
federal education legislation as the major criterion for evaluating educational 
relevance and impact) and substantial empirical evidence about what works and 
doesn’t work in supporting student achievement through contextual teaching 
and learning. Specifically, we need concerted, systematic research to help answer 
the following types of questions: How can the interjection of SCANS-like skills 
into the high school curriculum help increase student achievement in particular 
academic subject areas (e.g., reading, science, math)? Which assessment 
methodologies are most appropriate for students with particular learning 
strengths and styles? How does participation in contextual teaching and learning 
programs during high school affect what students do after leaving high school? 
Building a base of solid empirical research is no easy feat and could take a 
number of years to accomplish. 

The recommendations offered above constitute a tall order for industry-valued high 
school reform efforts. However, the end goal — improved teaching and learning for 
all high school students — constitutes strong motivation. 



Supporting High School Students Through Assessment of Academic and Industry-Valued Skills — Page 27 



References 



American Youth Policy Forum & Center for Workforce Development. (2000). Looking 
forward: School-to-work principles and strategies for sustainability. Washington, DC: 
Author. 

Ananda, S. M. (1999, April). Assessment in Career Education (ACE): The odyssey of a 
statewide student assessment system. Paper presented at Annual Meeting of the 
American Educational Research Association, Montreal, Canada. 

Ananda, S. M. (2000). How instructors can support adult learners through 
performance-based assessment. Paper commissioned by the National Institute for 
Literacy, Washington, DC. 

Ananda, S. M., & Rabinowitz, S. N. (2000). The high stakes of high- stakes testing. 

San Francisco: WestEd. 

Border, B. (1998). The status of alternative assessments through the 1990s: 
Performance and authentic assessments in relation to vocational-technical education 
technical skills , workplace skills , and related academic skills. Decatur, GA: The 
Vocational-Technical Education Consortium of States. 

Education Commission of the States. (2000, January). Advanced Placement courses 
and examinations (ECS Clearinghouse Notes). Denver, CO: Author. 

Elliot, J., & Knight, J. (2002). Education Daily , 35{ 3), pp. 1, 3. 

Herman, R., Aladjam, D., McMahon, P., Masem, E., Mulligan, I., Smith, O., 

O'Malley, A., Quinones, S., Reeve, A., & Woodruff, D. (1999). An educator’s guide to 
schoolwide reform. Washington, DC: American Institutes for Research. 

Kane, T. J., Staigar, D. O., & Geppert, J. (2001). Assessing definitions of 'Adequate 
Yearly Progress” in the House and Senate education bills. Los Angeles: School of 
Public Policy and Social Research, University of California, Los Angeles. 

Katz, L. G., & Chard, S. C. (1989). Engaging childrens minds: The project approach. 
Norwood, NJ: Ablex Publishing Co. 

Katz, R. H., Jackson, L. J., Reeves, K., & Benson, C. S. (1995). Urban career magnet 
high schools. In W. N. Grubb (Series Ed.), Education through occupations in 
American high schools: Approaches to integrating academic and vocational education 
(pp. 114-133). New York: Teachers College Press. 

Kaufman, P., Bardby, D., & Teitelbaum, P. (2000). High Schools That Work and 
whole school reform: Raising academic achievement of vocational completers through 
the reform of school practice. Berkeley, CA: National Center for Research in 
Vocational Education, Graduate School of Education. 




Supporting High School Students Through Assessment of Academic and Industry-Valued Skills — Page 28 



32 



Levesque, K, Lauen, D., Teitelbaum, P., Alt, M., Librera, S., & Nelson, D. (2000). 
Vocational education in the United States: Toward the Year 2000. Washington, DC: 
Office of Educational Research and Improvement, U.S. Department of Education. 

Long, C., & Crepeau, M. (2000). How to use projects in instruction and assessment. 
San Francisco: WestEd; and Washington, DC: Office of Educational Research and 
Improvement, U.S. Department of Education. 

Lynch, R. L. (2000). High school career and technical education for the first decade 
of the 21 st century. Journal of Vocational Education Research , 25, 155-198. 

Malyn-Smith, J., & Left, J. (1997). Scenario learning. Newton, MA: Education 
Development Center. 

Michigan State Board of Education. (1993). Employability Skills Assessment Kit, 
Version 1.0. Lansing, MI: State of Michigan, Department of Education. 

National Dissemination Center for Career and Technical Education. (2001, 
November). High stakes testing: Perspectives for career -technical education highlight 
zone: Research @ work. Columbus, OH: Information Synthesis Project. 

Rabinowitz, S. N., & Ananda, S. M. (2000). Balancing local assessment with 
statewide testing: Building a program that meets student needs. San Francisco: 
WestEd. 

Rabinowitz, S. N., & Ananda, S. M. (2001). Building a workable accountability 
system: Key decision points for policymakers and educators. San Francisco: WestEd. 

Rabinowitz, S. N., & Brandt, T. (2001). Computer-based assessment: Can it deliver 
on its promise ? San Francisco: WestEd; and Washington, DC: Office of Educational 
Research and Improvement, U.S. Department of Education. 

Rabinowitz, S. N., Zimmerman, J., & Sherman, K. (2001). Does high-stakes 
assessment affect student dropout rates ? Myth vs. reality. San Francisco: WestEd. 

Secretary’s Commission on Achieving Necessary Skills. (1991). What work requires 
of schools: The report of the Secretary's Commission on Achieving Necessary Skills. 
Washington, DC: U.S. Department of Labor. 

WestEd. (2001a). School-to-Career Assessment Gap Analysis Project. San Francisco: 
Author. 

WestEd. (2001b). Supporting data-driven decision making in schools: A toolkit. San 
Francisco: Author. 

Wills, J. (2002). Promoting new seals of endorsements in career technical education. 
Washington, DC: The National Association of State Directors of Career Technical 
Education Consortium. 

Wills, J., & Kaufmann, B. (in progress). National overview of skill standards. 
Washington, DC: Institute for Educational Leadership. 



Supporting High School Students Through Assessment of Academic and Industry-Valued Skills — Page 29 




U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 
Educational Resources Information Center (ERIC) 




NOTICE 



Reproduction Basis 



This document is covered by a signed "Reproduction Release (Blanket)" 
form (on file within the ERIC system), encompassing all or classes of 
documents from its source organization and, therefore, does not require a 
"Specific Document" Release form. 



X 



This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may be 
reproduced by ERIC without a signed Reproduction Release form (either 
"Specific Document" or "Blanket"). 



O EFF-089 (1/2003) 

ERJC 



